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ABSTRACT 

« 

Cloze tests and exercises are commonly used in language 
learning settings, and with recent interest in text manipulation, 
their implementation on computer has proliferated, as computer- 
generated cloze creation is purely mechanical. Performance on 
cloze exercises has been related to reading ability; however, it 
has been suggested (Windeatt, 1986) that strategies used by 
students in solving clozes on computer are quite different from 
those used by students working from print medium. 

This paper reports on a project aimed at finding out what 
students do when working cloze passages on computer. To generate 
data, a computer-based cloze program was configured so that all 
student keypresses would be recorded, and this was worked by 
several classes of students during their scheduled computer lab 
sessions. 

Besides confirming many of Windeatt' s findings regarding the 
differences in strategies employed when addressing computer- and 
print-based cloze passages (for example, that students working at 
computers solve problems in strictly linear order rather than 
working holistically) , the data reveal fresh insights into what 
students do when working with computer-based cloze. Following on 
these insights, suggestions are made for developers of CALL-based 
cloze programs, particularly in optimal design of help and hint 
features . 
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INTRODUCTION 

Cloze tests and exercises are commonly used in language 
learning settings. With recent interest in text manipulation, 
and as computer-generated cloze creation is purely mechanical, 
their implementation on computer has proliferated (e.g. Clozemas- 
ter . Jones, 1983; M-ss-ncr L-nks . Chomsky and Schwartz, 1984; 
Super Cloze , Millmore and Stevens, 1990). 

Performance on cloze exercises has been related to reading 
proficiency (Alderson, 1979 and 1980; Jonz, 1990); however, it 
has been suggested (Windeatt, 1986) that strategies used by 
students in solving cloze passages on computer are quite differ- 
ent from those used by students working from print medium. For 
example, students working at computers tend to deal with text one 
screen at time rather than in its entirety, to work linearly and 
"get stuck on particular gaps" (p. 89) , and to make more guesses 
e.t individual gaps than do students working cloze on paper. 
Windeatt also found that students using computer-based cloze were 
"reluctant" to use the help options available. 

As Windeatt 's experimental design employed split-screen 
video, with cameras trained simultaneously on the students and 
their monitors, the study points up salient differences between 
intrusive and non-intrusive studies. By "intrusive" is meant a 
study in which the researcher affects outcome by intruding on 
subjects with monitoring instruments not normally present during 
the process under study. Non-intrusive studies are those in 
which subjects are not aware that they are being studied. 

Some devices (for eicample, video or audio recording equip- 
ment) may have only a temporary effect on subjects, and this is 
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often taken into account; for example, Fawcett (1980) was careful 
to collect samples of children's informal speech from the point 
that the children seemed to ignore the microphone. Thus the 
intrusion may not persist throughout an entire experiment. 
However, there is no accurate way of determining when intrusion 
ceases to be a factor, if it ever does, nor when it might unex- 
pectedly come back into play during an experimental treatment. 

Thus intrusion becomes a factor in a survey whenever a sub- 
ject is aware that he or she is being studied, and it is diffi- 
cult to say how this awareness peaks and ebbs during an experi- 
ment. It is accordingly difficult to assess the spontaneity of a 
subject's actions in vitrio as opposed to those in vivo , as one 
could characterize conditions in non-intrusive studies. Aware 
that their moves are being recorded for replay, analysis, and 
judgment later, subjects may alter their behavior compared with 
what they would c* .> if they didn't know they were being studied, 
thus reducing confidence ir using research findings to predict 
behavior in circumstances beyond the experimental conditions. 

There are perfectly good reasons for doing intrusive studies, 
as when the researcher requires more control over variables (e.g. 
control over task and tima-on-task, recapitulation via inter- 
views) than would be possible with non-intrusive studies. Con- 
versely, researchers studying phenomena non-intrusively may lose 
these advantages, or lose the possibility for control treatment 
altogether. However, it may be that only non-intrusive studies 
can provide a true picture of naturally implemented CALL. 

One circumstance pertinent to CALL is its function as a tool 
for self-access, and the study of CALL effectiveness in self- 
access does not lend itself to intrusive study. In other words, 



if we want learners to act in a way natural to self-access, then 
the prime conditions of self-access (e.g. free choice of materi- 
als, work in private) must be maintained. Thus the study of CALL 
under self-access conditions cannot accurately be studied if 
subjects are aware that their behavior will come under scrutiny. 

In a prior non-intrusive study, Stevens (1990) tracked stu- 
dent interaction with computer-based Hangman and classified 
student approaches to solving the problems as being either compe- 
tency-based or non-competency based; that is, the subjects either 
used linguistic competence to solve the Hangman puzzles or they 
didn't. Common non-competency-based behaviors were abuse of 
hints, and resort to patterned or clustered keypresses. Stevens 
found that his subjects exhibited only 57% competency-based 
behaviors vs. 92% competency-based for "ideal" learners working 
either as native or non-native speakers of a target language; in 
other words, many subjects in the study employed Hangman as a 
self-access distraction. The study has not been replicated under 
intrusive conditions, but it is plausible that students who knew 
they were being studied would have paid more attention to what 
they were doing. Conversely, it is possible that Windeatt's 
finding that subjects made infrequent use of the help options 
would be overturned in a non-intrusive replication of his study. 
These are areas obviously ripe for future study. 

One question addressed in the present research is: do stu- 
dents "tune out" in a similar manner with computer-based cloze, 
which one would expect would be more cognitively engaging than 
Hangman, and in self-access might attract students of more seri- 
ous caliber and inclination? This paper reports on a nor,-intru- 
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sive study aimed at finding out just what students do when work- 
ing cloze passages on computer. 

PROCEDURE 

Super Cloze is a computer program which generates cloze 
exercises from ascii text files, one paragraph at a time, by 
default deleting every fifth word in the paragraph. However, 
students may over-ride the default setting as follows: they may 
change the deletion ratio to any value between 2 and 9; or they 
may choose to have only words in special word lists deleted. 
Super Cloze also offers a third option: deletion of words desig- 
nated as targets in the ascii text itself, but this facility was 
not available to subjects in this study. 

Super Cloze was placed on self-access in the Student Resource 
Centre (SRC) at Sultan Qaboos University in Oman and configured 
so that all keypresses of students using the program would be 
recorded. Occasionally, students were taken as a class to the 
SRC and encouraged to use the program with materials they were 
studying in that class. 

Data in this study were collected from one such session. 
This particular session was chosen because the researcher was 
present and therefore had some idea of who the subjects were and 
what they were doing while working the cloze lessons; thus it was 
possible to attenuate some of the disadvantages to non-intrusive 
studies. It should be stressed however that the role of the 
researcher during the session was to help students locate their 

« 

materials in the database; the researcher did not insist that any 
student use these materials or work only with Super Cloze . It 
was only after accumulation of a sizable body of information that 
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the need to work with a limited subset of the available data 
became obvious, at which point the researcher settled on this 
subset for the reasons just noted. 

The data therefore represent interaction with the cloze 
program of 28 Omani second year university engineering students, 
both male and female, grouped at 14 computers in a CALL lab 
scheduled during their ESL class. Although free to do as they 
liked, the students had been encouraged to use Super Cloze to 
practice a passage on Young's split screen experiment, a reading 
whose paragraphs were lengthy and difficult, though all the 
students were supposed to have read the material beforehand, and 
the experiment had been discussed in class. The students in the 
experiment spent from 10 to 45 minutes on the recommended materi- 
als, 30 minutes being about average. 

The data were analyzed in two stages. As often happens in 
exploratory studies, data analysis was more tedious than it might 
have been; in fact, the act of analysis itself led to many im- 
provements to the data collection program. One of these improve- 
ments was to build in more frequent time markers, so that in 
future more accurate information regarding timing can be taken 
than that noted above. Other improvements will cause most of the 
first stage analysis to be done automatically in future studies 
using Super Cloze . 

In the first stage analysis, the interaction was reproduced. 
The cloze passages were reconstructed, student cursor movements 
were tracked from gap to gap, student input at each gap was 
reconstructed, and a complete inventory of student behaviors and 
strategies was compiled. 
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In the second stage, the student behaviors and strategies 
were categorized as falling within one of four super-categories 
and tallied » The super-categories were: (1) achieving the cor- 
rect answer the first time, (2) achieving the correct answer 
after feedback to error, (3) utilizing help, and (4) skipping a 
problem (once attempted) without achieving the correct answer. 

Super Cloze was designed so that students should always be 
able to achieve a correct answer. Besides the program checking 
their answers for match with the deleted text, the students can 
see the number of letters required in any gap. Furthermore, they 
can at any time see the letter required at the current cursor 
position. Finally, they can see the solution; i.e. see the 
paragraph with all the gaps filled in and return to solving the 
problem. So options for achieving the correct answer range from 
trial-and-error through use of hints (to reveal any number of 
characters) to copying the answer from the solution screen, and 
these options are provided to prevent students' ever becoming 
"stuck" on any given problem. 

In other words, the program provides users with multiple 
options regarding the strategies they wish to employ in resolving 
answers to cloze gaps, and these strategies come into play when- 
ever subjects seek help, quit, or are not able to answer cloze 
gaps before first press of the Enter key. 

(1) ACHIEVING THE CORRECT ANSWER THE FIRST TIME 

Student input was considered correct -the- first-time when 
students either typed out the correct answer straight away or 
arrived at the correct answer after backspacing over errors or 
typos prior to first press of the Enter key for that problem. No 



attempt was made to analyze errors or typos when the student 
arrived at the correct answer before first press of the Enter 
key. 

(2) ACHIEVING A CORRECT ANSWER AFTER FEEDBACK TO ERROR 

If the correct answer and student input did not match once 
the Enter key was pressed, then the input was categorized as 
follows: 

(a) NONSENSE - the student input bore no relation to any word 

or plausible answer in English; 

(b) WRONG WORD - the student typed a wrong word; 

(c) WRONG FORM - the student used an incorrect form of the 

correct word; 

(d) CAPITALIZATION ERROR - the student's input did not match 

the expected answer only because incorrect ca«e was used; 

(e) DIACRITIC ERROR - the expected answer contained a number or 

diacritic not present in the student's input; or 
(e) PREMATURE ENTER - the student struck the Enter key prema- 
turely, before completing the answer. 

(3) UTILIZING HELP 

In the course ol getting an answer correct or incorrect, or 
prior to exiting without resolving a gap, students could get the 
program to provide help. There were three ways this could be 
accomplished: 

(a) HELP - Students could press Fl to see a help screen which 

explained how to use the program; 

(b) SEE SOLUTION - Students could press F5 to see the solution, 

which meant they were shown the paragraph in full - 
students could toggle between solution and problem 
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screens at will in this implementation of Super Cloze : 
(c) HINT - Students could press F8 to reveal the letter expect- 
ed at the present cursor position. 

(4) SKIPPING A PROBLEM WITHOUT ACHIEVING THE CORRECT ANSWER 

Students could exit a problem in three ways; either they 
could achieve a correct solution to a gap, or they could: 

(a) SKIP TO NEXT GAP - Students could press a cursor movement 

key to move on to another gap (students sometimes did 
this by mistake, pressing a cursor key in lieu of the 
Enter key). In this event, they had the option of re- 
turning to the gap later; or 

(b) SKIP TO NEXT PASSAGE - Students could press F9 to abandon 

the passage altogether. 

In addition to a tally of learner responses according to the 
above categories, data wore also collected on characteristics of 
the cloze passage itself. These data included the number of 
words in each paragraph presented, the number of gaps per pas- 
sage, the deletion ratio, and number of characters in each gapped 
word. Finally, where hints were used, the percent of the number 
of hints vs. word-length was calculated. 



RESULTS 

The data are arranged primarily by workstation, the 14 of 
which are numbered 1 through 9, then A through E, and can be 
considered paragraph by paragraph, as in Table 1 (1-1 in the left 
column means Group 1, paragraph 1, etc.), or gap by gap, as in 
Figures 1-6. The data are most revealing when presented in the 
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latter format; however, as there were 539 gaps attempted by the 
students at the 14 workstations over the 30 or 40 minutes most 
worked at Super Cloze , a table comprising these results is too 
large to present here. For convenience of presentation, the 
values for each paragraph can be totaled as shown in Table 1 (and 
explained more thoroughly in the Key to Tables and Figures) to 
give "bottom line" statistical information, from which certain 
conclusions can be drawn. 

The right-most column shows the options for targets for 
deletion chosen by students in each paragraph. Given the choice 
of deletion ratios (n=) varying from 2 to 9 (with 5 the default 
value) students invariably selected values ranging from 2 to 5 
(actual deletion ratios were slightly higher than these values 
because dates and numbers were not clozed) . Students could also 
chose word lists causing deletion of all pronouns, prepositions, 
helping verbs, all words with three letters, and words with 
greater than three letters (other choices were words of exactly 
or greater than 4 or 5 letters) . They also sometimes chose a 
list called "nothing," which deletes all the words in the passage 
(i.e. n=l) . Frequency of choice of the various options for 
deletion are shown in Table 2. 



TABLE 2 



TARGET FOR DELETION: 

every 5th word (n=5) 

helping verbs 

all words ("nothing"; n=l) 

every other word (n=2) 

pronouns 

3-letter words 

every 3rd word (n=3) 

every 4th word (n=4) 

more than 3-letter words 

prepositions 



TIMES CHOSEN: 



28 
8 
4 
4 
3 
3 
3 
3 
1 
1 
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It is encouraging to see that over half the time, the stu- 
dents experimented with something other than the default value. 
It is also possible to examine whether passage difficulty (as 
indicated by deletion ratio, average word length, number of 
letters in words attempted, passage length, and number of gaps 
per paragraph) had any bearing on student frustration or success 
(as indicated by number of gaps attempted, the percent of the 
entire passage attempted, and frequency of use of help features) . 

In general, correlations were not well established (Table 3), 
probably because there was not enough variety in the texts used 
in this study to yield appreciable differences in student success 
or frustration. However, at around the p ■ .005 level of signif- 
icance, the percentage of gaps completed by students per number 
of gaps in the passage correlated positively with deletion ratio 
and negatively with the number of gaps in the passage. In other 
words, and not surprisingly, the higher the deletion ratio (that 
is, the fewer the gaps) , the further along in the passage the 
students got. In addition, two tendencies (p < .1) were noted: 
the number of letters in gapped words encouraged use of See 
Solution and Hint; also longer word lengths suppressed the per- 
cent of passages attempted. 



DISCUSSION 

The data confirmed many of Windeatt's findings regarding the 
differences in strategies employed when addressing computer- and 
print-based cloze passages. For example, all but 3 of the 14 
groups of students worked the problems in strictly linear order; 
of these 3, two groups worked only one problem out of order (out 
of 19 and 32 gaps respectively) . The third group worked only 5 
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out of 40 problems out of order, all of these occurring in the 24 
gaps in the first two of three paragraphs attempted (A-l and A-2, 
Figure 1; the 16 problems in the last paragraph were all worked 
in linear order) . Thus there was a strong tendency for students 
to move down the screen from gap to gap, working each problem 
until it was resolved, rather than to approach the passages 
holisticully. 

Indeed there is scant evidence that students did any holistic 
reading at all. In the 58 paragraphs analyzed, only 9 (15.5%) 
were solved completely. Close examination of these cases shows: 

- in 4 of the 9 cases, the students saw the solution screen for 
59%, 58%, 34%, and 31% of the gaps they had to solve; 

- in one case (C-l, Figure 2), the students solved 17 of 26 
problems using hints, 9 of those (35%) using hints for more 
than half the characters in the answers, and 3 of the 9 using 
hints for ALL the characters in the answers; 

- in one case where the students solved 32 gaps with only 5 
wrong-word errors (5-2, Table 1), copying from the book 
cannot be ruled out; later in their session (5-6, Table 1) , 
this group solved 22 gaps correctly (no errors, only one See 
Solution) with the deletion rate set at every word, strongly 
suggesting that the book was used. 

- of the other three paragraphs solved completely, one had only 
1 gap and two had only 4 (because deletion from Wordlist had 
been selected, and only 1 or 4 of the words on the list 
appeared in those particular paragraphs) , in which case the 
reading required would be only in the immediate context of 
the gaps. 
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Therefore, in every case where students completed the para- 
graphs they were working on, they relied extensively on help; and 
in only three of the nine cases, where help used was in the 30- 
40% range, could it be said that reading of the entii-e paragraph, 
with comprehension, was a possibility. 

Each of the 58 paragraphs encountered was displayed in its 
entirety on the subjects' monitors, so that students in this 
study were confronted with 2435 gaps. Of these, the subjects 
attempted 539 gaps, or 22.1% (and solved 476, or 19.5% of all 
gaps encountered, correctly) . Thus the "reading" rate in this 
study would be about 20% of the total. 

In addition to the nine cases where students dealt in some 
way with all gaps in the entire passage, 8 paragraphs were com- 
pleted substantially beyond the 20% rate calculated above. 
Taking these on a case by case basis: 

- Group B worked only briefly on 4 of their 6 paragraphs; of 
the other two, they got through 50% of one (B-3, Table 1), 
but peeked at the solution screen 4 times in 12 gaps. In the 
other paragraph (B-6) , they solved 61% of the blanks, but in 
doing so used hints to discern ALL the characters in the last 
15 of 17 words attempted , echoing the "tuned out" non-compe- 
tent behavior discovered with the majority of the subjects in 
the Hangman study cited earlier. 

- Similarly, Group 8 solved 72% of their fifth passage (8-5, 
Figure 3) , but used hints in 11 of 22 gaps attempted. In 6 
of these, they used hints for 50% of the letters or fewer, 
but with 7 gaps (32%) , they requested to see all or all but 
one of the letters required (hints requested for the two gaps 
with just 2 letters are included in both tallies) . 
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- As can be seen from Figure 4, the work in D-l is scattered 
with errors eventually resolved, suggesting that some effort 
was put into the first 20 gaps. Although this group used the 
solution screen 7 times, they delayed use of this strategy to 
the last 10 of 30 problems attempted (at which point they 
were essentially copying answers) . They probably resorted to 
this ploy in order to get through most of the material by the 
end of the period, but reading strategies appear to have been 
applied in the first two thirds of the paragraph. 

- Similarly, Group 9 (9-1, Figure 5) concentrated on one para- 
graph throughout the entire class period and solved 78% of it 
with no use of help; however, a healthy scattering of errors 
suggests judicious use of feedback, implying application of 
valid reading strategies. 

- Group 6 (6-2, Figure 6) appears to have done some reading as 
well, completing 34% of the second of 5 paragraphs on which 
they made any real attempt. They correctly addressed 6 of 11 
gaps before using the solution screen and hints to solve 
three of the next 5 problems. Use of hints was reasonable: 2 
out of 5 and 4 out of 7 characters for two gaps. 

- Group E (Table 1) spent the entire time on two paragraphs, 
achieving 56% and 70% completion rates. In so doing, the 
students made only 11 errors in 55 gaps; here, the students 
were either especially competent, or (possibly) using an 
external text. 

Overall, these results suggest that reading rarely occurred 
on a global level, although reading of up to 20% of the material 
was not uncommon. Since the students tended to work linearly 
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from gap to gap starting with the first one, the gaps attempted 
would be the ones toward the beginning of each passage. Thas 
except in the cases noted above where students completed the 
passages or came close to completion, there is no evidence that 
the last 80% of the paragraphs were read at all. Windeatt's 
students in contrast read entire passages, as evidenced by re- 
cordings of their reading aloud. Whether they would have done so 
if left to their own devices is called to question by the results 
here. 

Also disputed in this study is Windeatt's finding that with 
CALL-based cloze, students "seem reluctant to use the 'help' or 
'cheat' facility" (p. 89). However, the Help information screen 
was requested at some point in 36% of all paragraphs in the 
present study. Hints were requested to help solve 9% of the gaps 
presented, and the students used See Solution for 18% of all 
words attempted (on average, 1.67 times per paragraph). As it is 
possible when viewing the solution screen to check on several 
gaps at a time, this facility may have been used to solve an even 
higher number of gaps. 

Requests for hints reveal a letter at a time and give stu- 
dents a boost in using their intelligence to work out a likely 
word fitting the context in question; thus some use of hints is 
to be encouraged. Despite the exceptions noted above, hints were 
rarely abused. If we define "hint abuse" as using hints to 
reveal over 50% of the letters in an unsolved word, then in 13 
paragraphs (22%) hint abuse occurred with at least one word; but 
in only 5 cases (8.6%) were more than a third of the gaps in a 
single paragraph thus abused. 

See Solution was meant to allow students to view the para- 
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graph in total and then return to the problem screen and concen- 
trate on vocabulary and syntax with questions of meaning re- 
solved, but the students in this study appear to have used this 
feature in many cases as a source of answers to be copied; thus 
this feature appears to have been abused. Indeed, in 14 of the 
58 paragraphs attempted (24%) , students invoked See Solution for 
a quarter or more of the blanks attempted in those paragraphs. 

As far as Windeatt's findings are concerned, it may be that 
the reluctance of his subjects to "cheat" was due more to the 
presence of cameras than to the nature of the medium under study. 
Indeed, this study shows that students are more willing to use 
help features when they feel they are "on their own" than when 
the researcher is palpably present. (Of course, this is true 
only if the students themselves are considered equal; it is more 
likely that individual differences in students could cause one 
individual or group to eschew help while another individual or 
group might view the ethics of receiving help differently. The 
presence of recording equipment could exacerbate such differ- 
ences) . 

An ancillary discovery made in the course of the present 
project is that capitalization can be taught with computer-based 
cloze. Of 28 problems where answers were incorrect only because 
of capitalization, 26 were resolved successfully; and this does 
not take into consideration cases where capitalized words were, 
once learned, reconstructed correctly by the students and counted 
simply as correct-first-time. Despite this success., the question 
remains whether it is worth frustrating students during the time 
it takes to drive home the point (note for example how A-l, 
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Figure 1, reached an impasse over capitalization and then revert- 
ed to nonsense input and skipping problems before quitting) . For 
diacritics, the answer is more assuredly no. Of the ten in- 
stances in the database where diacritics were encountered in 
gaps, only 5 such problems were resolved, whereas 5 were skipped 
in frustration. 

Getting back to the question raised by the Hangman study, do 
students "tune out" with computer-based cloze as was the case 
with Hangman? The answer appears to be: sometimes, but much less 
frequently. As has been noted, there was some persistent help 
abuse, suggesting that the subjects in those cases were giving 
little thought to solution of the cloze problems. Furthermore, 
at least one set of students used the text book to copy answers, 
and such instances appear in the data as sessions where answers 
are mostly correct, with very little interaction with the program 
and, presumably, very little reading for comprehension. 

On the other hand, the fact that the nonsense rate (the 
incidence of students entering gibberish) was a respectably low 
7.4% suggests that students approached computer-based cloze more 
seriously than they did Hangman, where incidence of throw-away 
key-presses was higher. Also, subjects gained some awareness of 
capitalization in Englishc In sum, although there was some 
evidence of lack of engagement on the part of students in the 
learning task, there was further evidence that 20% of the pas- 
sages - in some cases more - were being read. 

CONCLUSION 

Two lessons can be gained from the present research: 

(1) It is vital that CALL software be subjected to close scrutiny 



in order to verify its use in a manner appropriate to lan- 
guage learning. Many untested assumptions have been made by 
theorists in our field regarding the use of CALL software; it 
is important to test not only the assumptions (e.g. does text 
manipulation cognitively engage students?) but that the 
software carries out the assumptions (<?>g. how can a particu- 
lar piece of text manipulation software be improved so that 
it cognitively engages students?) . 
(2) It is important that research conducted to test CALL software 
for self-access be, at some stage, non- intrusive. Although 
carefully controlled and unavoidably intrusive studies have 
an important role in CALL research, it is important to be 
aware of their shortcomings, particularly in predicting the 
behavior of students who may not understand the rationale 
behind the software and may "tune out" as they use it on 
their own. 

This study has implications for the design of CALL-based 
cloze programs, particularly in optimal design of help and hint 
features. Higgins's (1983, 1988) vision of the computer as 
pedagogue to a bright and inquiring student did not consider the 
possibility that the student might be snapping his fingers and 
ordering the pedagogue about: answer this, answer that, and don't 
bother me with details. The concept is basically sound; it's 
just that the forthcoming pedagogue must not abrogate its respon- 
sibility to guide students toward self-reliance. Both this study 
and Stevens (1990) suggest that unlimited hint and solution 
features may do some students a disservice by providing them a 
tool with potential for abuse. Although such features are an 
integral part of self-access CALL, further experimentation is 



9 

ERIC 



17 20 



needed to refine exactly how and to what extent help can optimal- 
ly be meted out to students. 

This research has led to changes in the super Cloze program. 
Besides refinement of the data collection noted earlier, the 
program has been changed so that it no longer clozes words con- 
taining capitals or diacritics, with plans to give the student 
the option of attending to case if desired. The study has also 
pointed up the need for work on scoring as an incentive to pro- 
ductive use of the program (i.e. as a disincentive to abuse). 
For example, one hint per gap could be free, with the price in 
points sliding upwards and increasing dramatically as 50% of the 
characters in an unsolved word were revealed. A single peek at 
the solution screen could also be lightly charged, with subse- 
quent looks increasingly costly in point value. 

Computer-based cloze undoubtedly has great potential, but 
this study questions whether that potential is realized with 
present implementations. Developers of such programs need to 
establish patterns of actual use in order to build into their 
programs inducements to pedagogically viable outcomes. 
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KEY TO TABLES AND FIGURES 



w/p B words per paragraph 

#gap « number of gaps per paragraph 

d/r « deletion ratio (w/p divided by #gap x 100) 

#att * number of gaps attempted per paragraph 

Xatt = X of gaps attempted <#att divided by #gap x 100) 

For the following values, columns in Figures are TOTALED to give "bottom line 11 results unless otherwise noted. 

I en * length of each gapped word 

bottom line is avlen, or average word length 
Itr » total number of letters in a passage (sum of all len) 
#ht * nutter of hints per word; bottom line is total 
Xht = #ht divided by len in column 

bottom line: total #ht divided by total Itr in paragraph x 100 
help * Help screen seen # of times indicated 
SS * Solution seen for items indicated 
XSS - total solutions seen divided by #gap x 100 
xns = student input was nonsense 
xse * spelling error in student input 
xww * student input wrong word 
xwf = student input correct word in wrong form 
xce = case er«*or in student input 
xde * diacritic error in student input 
xpe = Enter key pressed prematurely 

exok « exited problem; OK - learner input matched expected answer 

skip = skipped problem by pressing cursor key 

exQt * exited problem by pressing F9 or F7 

Option = Option for deletion target selected, as follows: 

n= ... where number is deletion rate chosen 

noth NOTHING word list chosen 

prep PREPOSITION word list chosen 

hvrb HELPING VERB word list chosen 

pron PRONOUN word list chosen 

31 tr all 3- letter words deleted 

3pls all words > 3 letters deleted 
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TABLE 3 



Percent Number Number Help Percent 

attempted attempted of hints screen See Soln. 



Paragraph 
length 



.1690 
(.2089) 



.1058 
(.4337) 



-.0939 
(.4871) 



-.1269 
(.3468) 



.1347 
(.3179) 



Average word -.2823 -.1059 -.0353 .0896 .1795 

length (.0334)* (.4332) (.7946) (.5076) (.1816) 



Number of gaps 
per passage 
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(.0053)** (.5583) (.4903) (.3385) (.9723 



Number of letters 
in gaps per 
paragraph 



n/a 
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Deletion ratio 
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Coefficient (significance level); sample size = 57 
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